AD-A16@ GS@ PLAUSIBILITY FUNCTIONS OF IOWA VOCABULARY TEST ITEMS 

‘ ESTIMATED BY THE SIM.. CU) TENNESSEE UNIV KNOXVILLE DEPT 
: OF PSYCHOLOGY F SAMEJIMA DEC 84 RR-84-1-ONR 

UNCLASSIFIED N@@014-81-C-@569 F/G 5/18 


F— -4 
iS 
oa 


| 
Ie Ls 


= 
N 
N 


FEEEE 
il 


EEFERF 


fs 


(FFE 


F 
ig 
ter 


as 


N28 WES Ws. 


MICROCOPY RESOLUTION TEST CHART 
NATIONAL BUREAU OF STANDARDS - 1963-4 


eo eos 


J 


aoe 
. 


ee re ee ee 
ere Patt awe? 


, te Te sD”, 
ee we > 4 os eo ROS % 


ONR/RR-84-1 


(12) 


PLAUSIBILITY FUNCTIONS OF IOWA 
VOCABULARY TEST ITEMS 
ESTIMATED BY THE SIMPLE 
SUM PROCEDURE OF THE 
CONDITIONAL P.D.F. APPROACH 


AD-A160 850 


FUMIKO SAMEJIMA 


UNIVERSITY OF TENNESSEE 


KNOXVILLE, TENN. 37996-0900 D | IC : 


ELECTE 
DECEMBER, 1984 OCT 311985 _ 
f° 


Prepared under the contract number N00014-81-C-0569 
NR 150-467 with the 
Personne | and T- “Ree "Programs 


85 10 29 083 


Unclassified 
EES 
SECURITY CLASSIFICATION OF THIS PAGE (When Date Entered) 


REPORT DOCUMENTATION PAGE 


REPORT NUMBER 12. GOVT ACCESSION NO.| 3. RECIPIENT'S CATALOG NUMBER 


5. TYPE OF REPORT & PERIOD COVERED 


ONR/Research Rep 


4. TITLE (and Subtitte) 


Plausibility Functions of Lowa Vocabulary Test 
Items Estimated by the Simple Sum Procedure of 
the Conditional P.D.F. Approach 


Unclassified 
6. PERFORMING ORG. REPORT NUM@ER 


7 AUTHOR(s) 


8. CONTRACT OR GRANT NUMBER(2) 


Dr. Fumiko Samejima NQ0014-81-C-0569 


PERFORMING ORGANIZATION NAME AND AOORESS 10. PROGRAM ELEMENT, PROJECT, TASK 
Ase: AREA & WORK UNIT NUMBERS 
Personnel and Training Research Programs 


Office of Naval Research 
Arlington, VA 22217 


11. CONTROLLING OFFICE NAME ANO AOORESS 


NUMBER OF PAGES 
193 pages 


MONITORING AGENCY NAME & AOORESS (If different from Controlling Office) 15. SECURITY CLASS. (of thie report) 


Unclassified 


Sea, OECLASSIFICATION/ OCOWNGRAOING 
SCHEOULE 


DISTRIBUTION STATEMENT (of this Report) 


Approved for public release; distribution unlimited. Reproduction 
in whole or in part is permitted for any purpose of the United States 
government. 


OISTRISUTION STATEMENT (of the abatract entered in Block 20, if different trom Report) 


SUPPLEMENTARY NOTES 


KEY WOROS (Continue on reverse side if neceseary and identify by block number) 


Operating Characteristic Estimation 
Tailored Testing 
Latent Trait Theory 


20. ABSTRACT (Continue on reverse side if necessary and identify by block mamber) 


(Please see reverse side) 


| 


'cRM : 
3D . jan cy 1473 EDtTION OF 1 NOV 65 1S OBSOLETE 


Unclassified 
SoM O1D2- Fe ddd B04 : 


ay 
SECURITY CLASSIFICATION OF THIS PAGE (When Deta Entered) 


ar 


assified 


SECURITY CLASSIFICATION OF THIS PAGE (When Date Entered) 


Simple Sum Procedure of the Conditional P.D.F. Approach was 
applied for estimating the plausibility functions of the distractors 
of the Iowa Vocabulary Test items. In so doing, the normal ogive 
model was adopted for the correct answers of those items, and those 
items were used as the substitute for the Old Test. The group of 
subjects consists of 2,364 students who took the Level 11 tests in 
1971 through 1977. The results indicate the existence of informa- 
tive distractors for certain test items. The model validation study 


accompanied to it indicates that for most items the normal ogive 


model is suitable. ie eee Le al, a Chae. 


— 


ee af fs oo we , Ps ce ae fp ess 


} 


» N O102-¢F- 914-6001 


Unclassified 


re ne TY 
A ee Aone 2) cr eres, Mok Mime Mei meme) Ll? See oT Tee 


TABLE OF CONTENTS 


Page 
Abstract 
I. Introduction 1 
II Iowa Test Data 3 
III Simple Sum Procedure of the Conditional P.D.F. 5 
Approach and Normal Approach Method 
IV Old Test 10 
V Item Parameter Estimation for the Old Test Items 12 
Following the Normal Ogive Model 
VI Test Information Function of the Old Test and the 24 
Transformation of the Latent Trait 6 to tT 
VII Estimated Plausibility Functions of the 32 
Distractors 
VIII Model Validation 61 
IX Discussion and Conclusions 68 
References 70 
Appendix 72 
Accession j Por 
NTIS GRAGL 
DTIC T:5 
Unarnecunces 
a ie a 
By 7 a 


Distritution/ | 
Availability Cadas | 


$ 


QUALITY 
INSPECTED 


Pe ae 
Mri 
5 
b- 
: 
Ko. 
a 


PLAUSIBILITY FUNCTIONS OF IOWA VOCABULARY TEST ITEMS ESTIMATED BY 
THE SIMPLE SUM PROCEDURE OF THE CONDITIONAL P.D.F. APPROACH 


ABSTRACT 


Simple Sum Procedure of the Conditional P.D.F. Approach combined 
with the Normal Approach Method was applied for estimating the 
plausibility functions of the distractors of the Level 11 Vocabulary 
Subtest items of the Iowa Tests of Basic Skills. In so doing, the 
normal ogive model was adopted for the correct answers of those items, 
and those items were used as the substitute for the Old Test. The 
group of subjects consists of 2,364 students who took the Level 11 
tests in 1971 through 1977. The results indicate the existence of 
informative distractors for certain test items, and it is confirmed 
that most items belong to the Informative Distractor Model rather than 
the Equivalent Distractor Model. The model validation study accompanied 
to it indicates that for most items the normal ogive model is suitable 


for their correct answers. 


The research was conducted at the principal investigator's laboratory, 
405 Austin Peay Hall, Department of Psychology, University of Tennessee, 
Knoxville, Tennessee. Those who worked for her as assistants include 
Paul S. Changas, Vicki R. Newton, Mehrdad A. Saravi, Deborah Wichlan 
and Cindy Wheatley-Lovoy. 


I. 


Introduction 
Three-parameter logistic model (Birnbaum, 1968) has been 
ons popular among researchers who deal with multiple-choice test items. 
The model is based upon the knowledge or random guessing principle, 
which assumes that the examinee either knows the answer or guesses 
randomly. In that model, each item is scored either right or wrong, 
vy depending upon whether the examinee has chosen the correct answer or 
one of the incorrect alternative answers. It should be noted that 
following the three-parameter model each incorrect alternative answer 
‘ is treated as equivalent to each other, and it is implicitly assumed 
that, owing to the subjects’ random guessing behavior, those 
distractors have identical operating characteristics. We will not 
4 lose any information, therefore, even if we recategorize all the 
incorrect alternative answers into a single category, and assign zero 
as the item score to those who have chosen any one of the distractors. 
B Such a family of models belongs to the Equivalent Distractor Model. 
Same jima has proposed a new family of models for the multiple- 
* choice test ttem (Samejima, RR-79-4), which assumes that the examinee 
chooses one of the distractors for his answer more or less 
intentionally, and, therefore, each incorrect alternative answer, as 
well as the correct answer, provides us with its unique information. 
In those models, the examinee’s random guessing behavior is still 
taken into consideration, but only as the last resource when he has 
no idea as to which alternative is more plausible as the answer to the 


\ question. The plausibility function of each distractor, which is the 


conditional probability assigned to the choice of that particular 
distractor, given ability, is the information source. If a model in 
the family fits our multiple-choice test data, then we shall be able 
to estimate the examinee’s ability more accurately, using information 
obtained from the distractors as well as from the correct answers. 
Thus the multiple-choice test item is no longer a "blurred image" 

of the free~response test item, but it has a unique status as a 

test item which provides us with information that the free~response 
test item is not able to. Such a family of models belongs to the 
Informative Distractor Model. 

It will be worthwhile to estimate the plausibility functions of 
the distractors of existing test items, to find out if, indeed, each 
distractor provides us with its unique information. In so doing we 
need some nonparametric method of estimating the operating 
characteristics, for we have no idea what type of mathematical 
function each plausibility function follows. Molding it into some 
parametric form from the very beginning will be harmful, therefore, 
more than helpful. 

In the present study, a subset of the data collected on the 
Iowa Tests of Basic Skills was used as "existing data". It is based 
upon the Vocabulary Test items for Level 11. The whole set of data 
was obtained with the courtesy of Professor William Coffman of the 
University of Iowa, and was introduced in a previous research project 
(Same jima and Trestman, RR-80-1). For brevity, hereafter, we shall 


call the whole set [Towa Test Data. Its brief description will be 


ae” 


given in Section 2. 

In estimating the plausibility function of each distractor of 
each test item, Simple Sum Procedure of the Conditional P.D.F. 
Approach combined with the Normal Approach Method was used (cf. 
Samejima, Final Report, 1981). A brief description of the Simple 
Sum Procedure of the Conditional P.D.F. Approach and the Normal 
Approach Method will be given in Section 3, in comparison with other 


methods and approaches developed by the author. 


Il. Iowa Test Data 

Iowa Test Data are based upon the Iowa Tests of Basic Skills, 
Form 6, Levels 9-14 (Hieronymus and Lindquist, 1971). These tests 
have been designed, constructed and revised at the College of 
Education of the University of Iowa since 1935, with the general 
school population in mind. Each of the level numbers, 9 through 14, 
corresponds to the age of subjects who are suitable for the subset of 
test items that specific level contains. Thus the tests are designed 
basically for the fourth through ninth graders. 

There are eleven tests in the battery, each of which focuses 
upon a different basic skill. The numbers of test items in the eleven 
separate tests vary within the range of 74 through 178, including all 
the six levels. Table 2-1 presents these eleven tests, the numbers of 
test items and some other information related to their 


administration. 


The data were collected in three different school systems In 


TABLE 2-1 


Iowa Tests of Basic Skills, Form 6, Numbers of Items, 
Administration Sessions and Time Limits. 


Adainistration 
Session 


Spelling 
: Capitalization 
-3: Punctuation 

: Usage 


Second Session 
80 Minutes 


: Wap Reading 

W-2: Reading Graphs and 
Tables 

W-3: Knowledge and Use of 

Reference Materials 


Third Session 
85 Minutes 


Fourth Session 
65 Minutes 


Mathematics Concepts 
M-2: Methematics Problem 
Solving 


TABLE 2-2 


Number of Items in Each of the Three Subtests of Each of the 
Eleven Tests and in Total for Each of the Three Levels, 11, 
12 and 13. 


82 133 82 73 73 53 67 48 99 90 61 


the State of Iowa, in the years 1971 through 1977, using the subtests 
of Levels 11, 12 and 13. Table 2-2 presents the number of test items 
contained by each of the three levels, for each of the eleven tests. 
Since there are overlapping test items between two, or even three, 
adjacent levels, the total number of items in each test is less than 
the sum of the numbers of items in the three separate subtests. 

In the original data, there are 2,460 examinees who took the 
subtests of Level 11, 2,452 who took those of Level 12, and 2,527 who 
took those of Level 13, to make the total number of examinees 7,439. 
With close examination of those original data, however, certain 
examinees who omitted too many test items were excluded, and the 
resulting “revised data" are based upon 7,246 examinees in total, 
with 2,364, 2,413 and 2,469 for those who took the three separate 
levels of subtests, respectively (cf. Samejima and Trestman, RR-80-1). 

In the present study, the results of the 2,364 examinees on the 
Level ll Vocabulary Subtest were analyzed. The choice of Level 11 was 
more or less arbitrary, but the Vocabulary Subtest was selected more 
intentionally, due to the factor analysis result which will be 
introduced in Section 5. This subtest has forty-three test items, 
each of which has four alternative answers, i.e., one correct answer 


plus three distractors. 


IIlL. Simple Sum Procedure of the Conditional P.D.F. Approach and 


Normal Approach Method 


Same jima’s methods and approaches of estimating the operating 


characteristics of discrete item responses were introduced in previous 


=6= 


research reports (Samejima, RR-77-1, RR-78-1, RR-78-2, RR~78-3, 
RR-78-4, RR-78-5, RR~78-6), and were summarized later (Samejima, Final 
Report, 1981). They are characterized by two features, i.e., 
1) estimation is made without assuming any mathematical forms for the 
operating characteristics of discrete item responses, and 
2) estimation is efficient enough to base itself upon a relatively 
small set of data of, say, several hundred to a few thousand 
examinees. 

There are two main approaches, one of which is called Bivariate 
P.D.F. Approach, and the other is called Conditional P.D.F. Approach. 
In the former approach, we approximate the bivariate distribution 
of the transformed latent trait +t and its maximum likelihood 
estimate + , for each of the subgroups of examinees who share the 
same discrete item response to a specified item. Thus the procedure 
must be repeated as many times as the number of discrete item response 
categortes for each item. In contrast to this, in the Conditional 
P.D.F. Approach, this is done for the total group of subjects. 
Effort is focused upon the approximation of the conditional 
distribution of T , given t » for the total group of examinees, 
and then the result is branched into separate discrete item response 
subgroups for each item. This latter approach is further categorized 
into three procedures, i.e., Simple Sum Procedure, Weighted Sum 
Procedure and Proportioned Sum Procedure. In each approach, there are 
three methods of approximating the conditional distribution of 1c, 


given c » and they are called Pearson System Method, Two-Parameter 


Beta Method and Normal Approach Method, respectively. 


It is appropriate to say that, theoretically, Bivariate P.D.F. 
Approach is a more orthodox approach, and Conditional P.D.F. Approach 
1s a simplified version of the former. The latter approach has two 
big advantages, however, in the sense that the CPU time required is 
substantially less, and that it does not have to deal with subgroups 
of small numbers of subjects in approximating the joint bivariate 
distributions of T and T . It is also appropriate to say that 
both Two-Parameter Beta Method and Normal Approach Method are simpler 
versions of Pearson System Method; and yet the latter two methods 
have an advantage of using only the first two estimated conditional 
moments of Tt , given t » whereas the former requires the additional 
third and fourth conditional moments, whose estimations are less 
accurate compared with those of the first two conditional moments. 

Our past experience with simulated data indicates that in many 
cases even by using the combination of relatively simplified approach 
and method we can estimate the operating characteristics fairly 
accurately. For this reason and because of the advantages described 
in the preceding paragraph, our choice of approach and method in the 
present study is the combination of the Simple Sum Procedure of the 
Conditional P.D.F. Approach and the Normal Approach Method. 

Let 98 be ability, or latent trait, which assumes any real 
number, and be any discrete response category to item g . We 


assume that there is a set of test items measuring 0 whose 


characteristics are known. This set of test items is called Old Test. 


Let I(6) denote the test information function of the Old Test of on 


such items. The transformation of 9 to t+ is made by 
~1 1 
(3.1) t= C, | [r¢t) 1/2 at + Co > 


where Cy is an arbitrary constant for adjusting the origin of 
t , and Cc, ig an arbitrary constant which equals the square root 
of the test information function, I*(t) , of T , so that we can 


write 
(3.2) c- [ra(r)]i/2 


for all t . This transformation will be simplified if we use a 
polynomial approximation to the square root of the test information 
function, 1I(6) , which is accomplished by using the method of 


moments. Thus (3.1) can be changed to the form 


m 
(3.3) t & co £ a, (kti)! oektl 4 ¢ 
gg 0 
mtl 
= £ ar aX ‘ 
k=0 


where (=0,1,...,m) is the k-th coefficient of the polynomial of 


degree m approximating the square root of I(@) , and age is the 


new k-th coefficient which is given by 


k= 0 


= (Cyk)! ap) Kk = 1,2,00¢,mtl « 


The first through fourth conditional moments of 1 , given i. » can 
be obtained from the density function, g*(T ) , of T and the 


constant C, by the following four formulae. 


a rs = d A 
E(t |T.) = tC, 2 log a(t.) A 


dt 
s 


Z eS Pe 2 A 
Var.(t|t_) = C 2t1 + C 2d" log g(t_)] . 
Ss a 2 S 
dt 
s 
ere ee ~6; d3 ae 
E{{t-E(t|t_)}-|t_] = cyl log g*(t )] . 
Ss s at Ss 


. , ss a . 
El{r-E(t|@_)} 1) = C713 + 6C,-27¢ 9 log gz) 
s Ss a 2 s 
dt 
S 
a2 4 


a 


dt 
s 


log gt(i_)}? + c,~*{ 
2 s 


+ 30,744 log g*(z hd - 
S 


dt_* 
s 
Note that in the above formulae the first moment is about the origin, 


while the other three are about the mean. The two coefficients, BF 


and B, » and Pearson’s criterion « are obtained by 


(3.10) 


s (3.11) <= AC H43)714(2 8-34-6906 H-3 QI! 


by substituting u, , u, and u, by Var.(t|t_), 


El{r-E(r [7 17] and BU rmE(t [7 17] » respectively, which are 


obtained by formulae (3.6), (3.7) and (3.8). 


In the Simple Sum Procedure of the Conditional P.D.F. Approach, 


the operating characteristic, Py (6) , of the discrete item response 
8 


ke of an "unknown" item g is estimated through the formula 


N 
E grl[t 2 origi, 
Sek 2 s=] = 


(3.12) Peg = Eee? = 


where s (#1,2,...,N) indicates an individual examinee, and o(t|t.) 


a denotes the conditional density of tT , given te . This 


conditional density is estimated by using the estimated conditional 


moments of T , given - - In the Normal Approach Method, o(t|t.) 


is approximated by the normal density function, using the first two 


estimated conditional moments of Tt , given T. » which are given 


by (3.5) and (3.6), respectively, as its parameters. 


Iv. Old Test 


In this study, we need some suitable substitute for the Old 


Test, since there is no set of test items measuring the same 


vocabulary ability whose characteristics are known. To handle this 


situation, we use the Level 11 Vocabulary Subtest itself twice, i.e., 
first as the Old Test and later as the set of "unknown" test items. 
On the first stage, each item is rescored as "right" or 
"wrong", 1.e., dichotomously, and the normal ogive model on the 
dichotomous response level is assumed. We accept this model 
tentatively, and the item parameter estimation is made for each of 
the forty-three test items. On the second stage, these forty-three 
test items are treated as they are, i.e., as multiple-choice test 
items with polychotomous item responses. The "unknown" operating 
characteristic is to be estimated, therefore, for each of the four 
alternative answers of each item. As the result, we will obtain the 
estimated plausibility functions of the distractors for each item. In 
addition to this, the estimated operating characteristic of the 
correct answer of each item is compared with the hypothesized normal 
ogive function as a part of the model validation process, which will 
be Lntroduced in Section 8. If the model is validated, then we will 
accept the estimated operating characteristics of the incorrect 
alternative answers as the estimated plausibility functions of the 
distractors. If not, we will examine the invalidated test items, and 
either assume more suitable models for them or discard them, to 
produce a new Old Test and repeat the estimation process all over 


again. 


The item characteristic function, P,(6) , of item g in the 


normal ogive model is given by 


Rtg te tet ee Oe eo Ae 


—ho= 


a, (8-b,) - 
(4.1) P(e) = (20) 7} /2 [2 . e 


where ee is the item discrimination parameter and b, is the item 


difficulty parameter. 


V. Item Parameter Estimation for the Old Test Items Following the 


It is assumed that the response tendencies of our 2,364 
examinees behind the forty-three items of the Level 11 Vocabulary 
Subtest have a multinormal distribution as their joint distribution. 
If there exists a single common factor behind these forty-three 
response tendencies, then we shall be able to operationally define the 
factor as the vocabulary ability for the contents of this specific 
subtest. As the result of this assumption, the ability distribution 
for these 2,364 subjects will also be normal, and we shall define the 
origin and the unit of the ability scale as the mean and the standard 
deviation of this normal distribution. 

The tetrachoric correlation coefficient is obtained for each 
pair of test items, using the program written by the author. The 
resulting inter-item correlation matrix is given as Table 5-1. Those 
tetrachoric correlation coefficients were adjusted for the 
unbiasedness, which means that each value is a little less in the 
absolute value than the straightforward sample correlation 
coefficient. The item numbers in this table, which range from 24 to 

66 , are the same as those used in the actual Iowa Vocabulary Test. 


We can see in this table that all the correlation coefficients are 
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non-negative, the fact which indicates the existence of a strong, 
dominating common factor. This inter-item correlation matrix, R , 
was subtracted by the covariance matrix of the unique factor matrix, 
V , and the resulting matrix was factor analyzed, using the 

computer program for principal factor solution in Biomedical Computer 
Programs Multivariate Analysis Series 4 (BMDP4M). Actually the matrix 
(RV) was obtained by a usual iterative estimation of each of the n 
communalities, with the squared multiple correlation of each variable 
with all other variables as its initial estimate. 

The same procedure was applied for each of the other ten Level 
ll Subtests, and the resulting sets of eigenvalues are shown in 
Table 5-2*, except for those of the Level 1] Reading Comprehension 


Subtest (R)**. Inspection of this table tells us that Vocabulary 


*The computer program is written in the way that we can adjust the 
accuracy of the estimation of correlation coefficients in certain 
ways. After one way was applied for all the subtests, it was redone 
for Vocabulary Subtest by increasing the accuracy for a certain range 
of correlation coefficients, and the new result was used in the 
present study. Although the discrepancies are small, in this sense 
the results of the other nine subtests are not exactly comparable to 
the result of the Vocabulary Subtest. Also, the numbers of common 
factors used for the reestimations of the communalities are much less 
for the other subtests than 40 which was used for the Vocabulary 
Subtest, t.e., 15 ,15,13,15,10,15,20,15 and 15, 
respectively . 


**For this specific subtest, we obtained the message that the matrix 
is not positive semi-definite. It is suspected that there are a 
substantial number of zeros in the correlation matrix because of 
the unblasedness adjustment, and this may be one of the reasons for 
the message. 


TABLE 5-2 


LEigenvalus of the Matrix (R-V) for Each of the Ten Level 11 Subtests 
Obtained As the Results of the Principa! Factor Solution of Factor 
Analysis. 


14.6176 12.3175 10.5823 11.5618 8.4561 7.8066 6.5457 15.1236 10.2474 7.2963 
1.0398 1.5332 1.9527 2.1356 1.5431 1.9619 1.0297 4.2759 1.4570 1.4043 
0.7704 1.0122 1.5139 1.6246 0.9682 0.86137 0.9838 0.9698 0.957% 0.7383 
0.6788 0.82468 1.0949 4.2001 0.7260 0.7645 0.7495 0.8879 0.7146 0.6322 
0.6395 0.7283 0.7070 0.9655 0.7014 0.5331 0.6671 0.8028 0.6729 0.5763 
0.6288 0.6740 0.6257 0.7772 0.5827 0.46974 0.5789 0.7542 0.5537 0.5334 
0.6023 0.6048 0.61346 0.7368 0.5389 0.4299 0.5468 0.6152 0.5184 0.4784 
0.5512 0.8207 0.5439 0.5810 0.4632 0.4107 0.4518 0.5983 0.4542 0.4523 
0.5248 0.4739 0.4916 0.4825 0.4486 0.3541 0.3707 0.5493 0.4138 0.4165 
0.5084 0.4207 0.4478 0.6174 0.3925 0.3069 0.3143 0.5262 0.4038 0.3271 
0.4801 0.3885 0.3990 0.3530 0.3589 0.2329 0.2667 0.5190 0.3784 0.2913 
0.4658 0.3780 0.3545 0.3239 0.3332 0.2237 0.2318 0.4880 0.3544 0.2609 
0.4471 0.3330 0.3434 0.2952 0.3009 0.1719 0.2151 0.4504 0.3372 0.2399 
0.4133 0.2928 0.3209 0.2158 0.2859, 0.1617 0.18746 0.4165 0.3239 0.2045 
0.3966 0.2682 0.2981 0.2042 0.2093 0.1427 0.1487 0.3635 0.2803 0.1736 
0.3725 0.2069 0.2351 0.2022 0.1165 0.1236 0.0849 0.3516 0.2060 0.1218 
0.3537 0.2026 0.2123 0.1505 0.1301 0.1139 0.0508 0.3268 0.1943 0.0705 
0.3466 0.1783 0.1903 0.1387 0.0977 0.1005 0.0406 0.3018 0.1692 0.0645 
0.3188 0.1597 0.1435 0.1329 0.0747 0.0761 0.0236 0.2837 0.1475 0.0384 
0.3065 0.1354 0.1409 0.1015 0.0638 0.0629 0.0183 0.2500 0.14628 0.0237 
0.2673 0.1082 0.1127 0.0892 0.0544 0.0441 0.0087 0.2042 0.1339 0.0228 
0.2576 0.0972 0.0913 Q.0702 0.0391 0.0177 -0.0112 0.1980 0.1002 0.0153 
0.2413 0.0873 0.0893 0.0507 0.0277 0.0070 -0.0228 0.1910 0.09467 -0.0091 
0.2286 0.0742 0.0706 0.0271 0.0095 -0.0086 -0.0631 0.1778 0.0768 -0.0218 
0.2161 0.0645 0.0512 9.0176 ~0.0077 +0.0324 -0.0758 0.1665 0.0681 -0.0392 
0.1950 0.0478 0.0332 0.0118 +~0.0103 -0.0515 -0.0793 0.1566 0.0517 0.0452 
0.1800 0.0381 0.0051 -0.0163 +0.04682 -0.0721 0.1391 0.0331 -0.0594 
0.1698 0.0236 -0.0066 -0.0312 +0.0656 -0.0955 0.1190 0.0279 -0.0903 
6.1525 0.0180 <-0.0174 -0.0363 -0.0828 -0.1027 0.1059 -0.0025 -0.0991 
0.1402 0.0060 <-0.0329 -0.0675 -0.1108 -0.1164 0.0990 -0.0137 
0.1285 -0.0180 -0.0476 -0.0584 ~-0.1222 -0.1201 0.0837 -0.0306 
0.1216 -0.0271 -0.0701 -0.0659 ~0.1529 -0.1378 0.0686 -0.0394 
0.1139 -0.0385 -0.07467 -0.0851 -0.1500 0.0888 -0.0685 
0.0939 -0.0705 -0.1062 -0.0965 70.1713 0.0437 -0.0808 
0.0844 -0.9783 -0.1241 -0.1117 70.7846 0.0362 -0.0982 
6.0605 -0.1021 -+0.1373 -0.1298 -0.23570 0.0184 -0.1098 
0.0508 -0.1103 -0.1500 -0.1540 0.0757 0.1244 
0.04601 -0.1223 -0.1798 -0.1731 0.0083 -0.1458 
0.0172 -0.1322 -0.2007 -0.1923 “0.0138 -0.1598 
0.0060 -0.1637 -0.2288 -0.2152 “0.0221 -0.14691 

~0.0121 -0.1761 “0.0296 -0.1956 

“0.0154 - 3.1950 -0.0397 -0.2323 

~0.0275 -O0.2147 70.0625 

“0.0492 
0.0650 
~0.0788 
“0.0884 
-0.0954 
0.1259 
“0.1422 
-0.1520 
0.1637 
-0.1676 
~0.1872 
~0.2052 
-0.2239 
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Subtest is one of the few subtests whose second largest eigenvalue is 
substantially less than the first and also not so much greater than 
the third and other negligibly small eigenvalues, the fact which 
practically indicates a single common factor structure. This may be 
due to the fact that reading ability is always required in the 
performance in any subtest in addition to the content of measurement, 
but, unlike in the other nine subtests, in Vocabulary Subtest those 
two abilities are close in nature. In any case, this result is one of 
the reasons why Vocabulary Subtest was selected for the present study. 
The factor loading matrix of the forty-three response 
tendencies is given as Table 5-3. We can see in this table that all 
the factor loadings on the first common factor are positive, and, 
except for those of items 24 and 44 , they are greater than 0.300 , 
ranging from 0.316 for item 39 to 0.691 for item 30. The 
largest cluster of factor loadings we can find in those common factors 
excluding those in the first one is the pair in the fourth factor, 
i.e., 0.393 for item 33 and 0.368 for item 44 . Most of the 
factor loadings on those other common factors are less than 0.300 in 
absolute value, the fact which indicates their weak influences. From 
this result, a decision was made to proceed by defining operationally 
the first common factor as the vocabulary ability and to use the whole 


set of items in the Subtest as the Old Test. 


The proportion correct, Pg ? for each item g is given in 
Table 5-4. In the same table, also presented are the normal deviate, 


Yg » corresponding to the proportion correct Pg » and the estimated 


~~ Ttbe 


Poe i 


stb tur Tel *o- OTl’s oor 2e0°S 2i6°O ét¢d°u 9O2°O6 eo] 
clita sv0°U tul*c- 660°0- 902°0 0240°O0- 090°O O1T’o 6B1°O 2e1 
todru yt l°o- 91O°%O- o9t°0 1¢0°0 Sx (°G- 90T° O- 942°6 901°O T¢] 
bed tur zturd 65U°O- 206°0- o2t°o ¥10°C- ¥00°O $22°OG 290°9O ce! 
uu run + 20°O #e1°a 920°0- 060°0 20T°o- 160° 0- 642°0 1£60°O- ec! 
Cc ee 2uu°0- £CieG- o2t°c o1l0°0- 2c00°0- 980° 0- 2¢20°o 4£€2°O eel 
fab* crud ot0° O- 400°0- 04 6°O- BLI°o- £0T°O- bE T*O 2560°O 4€)} 
tou a= Pvo°o- 911°O 1e0°O- L121°0- 910°O- 410° 0- 412°0 090°0 9€1 
Gui “b&b 90°C $20°O- £22°0- 002°0 L8T°G 210°0- €90°O- %02°0O stl 
gf UR- uol?o- Giu0°U- 800°O 6t0°0- Z29T°0 560°0 290°0- $92°O >E! 
dry br lO sb0°O f€er°o- G60°0O- 6S0°0 400°0- 3%0°O 191 °O ttl 
GIL tun Tyl*o dkr PU 400°G- $20°0 z10°0- 0S0°0 2s0°0- $00 °O 2c 
buLU e¥T*O- €lo*cu- ett o- $40°O0 $10°o ezt°o 040 °0- 2i2°0 ted 
Ocu*y Lyz*o 710°o 2ot°o- £01°0 zZi0°0- Svt°o- 9271°0- 680 °O cel 
alotu- 09U"U 9ou°O- 990°0- £00°0- 28 2°0- €60°0O $90°0- €50°O 621 
orb eo- + 60° O sul’ o- 86t°d $01°0- 6%72°O t10°O 91t0°0O- 98C°0- u2l 
duutu rL0°0O- Tuu?0- 9%2°0 T10°0 ott o- $22°o- Tlilto- ¥90°O d21 
tulo- €kU7O i0t°o 910°0 €uc°od 4e0°0 260° 0- T20°O t€1°o- 921 
%10°%G 1c0°O- 920° 0- 420°0 120°O 240°0 1tG°0 $ot°o- sol°o se} 
vere ter o- 62t°oa S$10°0- 8%0°O 600°0 212° 0- evi? o- 120°O “21 
usbeu- 670° O OoOT*O Z90°0 t£u°o 4£50°0 ¥cO°O- 890°%O- 040°O €21 
2g ]%u- deyu*0 Te v*0 220°0- $20°0- €%0°G- 890° 0- £80°%0O- 9uT°O- 22k 
cvuro isto val’o €22°0 9»0°0 9ET°O B9t°0 9S0°0- 262°O 12! 
WlotG tul"O booed by9T°O Ts0° 0- 900°0 040°0- €tG°o Gol°*o- Cet 
9livu y40°O 0v00°O- 020°O ¥€2°0- 990°0- z224°0 8i0°O 320°O Cam | 
Ubt tu Lriro *U1T°U €60°O g22°0 €eteo $t0°0- e01°O $92 °0- 6Ul 
eG ttau 370° O yGh?o 620°0- e2t°o- 260°0 600° 0- 60T*Q- S972°a aut 
lou? G 70° 0 L9u°d ¥T°O- £90°0 €40°O €£0°0 Z210°0- 4t0°O 911 
ebb %7 t¥¢d°O byo*C- 2461°0 z80°O- 2Ub°o- $90°0- ¥00°O 0e80°0O $l 
wu0"O ctt°o £20°G- £90°0 Lst°o0- z200°0- %60°0 z10°0 6tG°o 4ll 
l2g"u b50°O 621° O0- Stl’°o- T21°o z0°0o €$0°0 800 °0- cuz °o- etl 
Pe Sener) 9st°O- 0O9T°O 19T°O 620°0O ore °O e2T°o- S90°0- etl°o- el 
culeu 794° Q- 21i°0- 100°0 900°0 620°0O T2z1°O €00°0- Oo12°o- tl 
dur "yu sil°o- 912°O $60°0 €90°O 292° 0- f£6b°O ¥20°O- 0S2°0- Ci 
dul 9LU°G- 600°0- 950°O 190°0- 8S0°O 220° 0- 00°90 €i2°0- 61 
80 °U slu°O- 4310°0O ©00°0 z2tt°o- 4€0°0- 27$0°0 £22°O0- 960°O eo 
OGU%uU- vi0°O- vl0°O- 901 °0- 2s0°0- 160°0- 6Ct°O- 180°0- €20°OC il 
SSIES ccurv £92°0- 2460°0- 610°0- s91°O 4¥10°O- ¥Z20°O 99T°0- 91 
SST°uU- cbu*Q- $9u°0 902°0- 24%t°O- 4i10°0 291°0 600°0O sOT"0- $s] 
Oot -u- 1oG°0O BET? O- 190°0- 72°00 $40°0- ot1°o 221°0- 661 °0- $l 
LU U vwUucO- $10°0- oo°o- s1o°o 4it°o- 220°0- 862 °0- 2%0 °O- Cr 
040°0O- édu°O- 420°0O- 9i0°0- Gilto- $90°0 9%0°0O 601°0O- 1SG°0- et 
GOI" vurued 200°O Tt2°0- 0€ 2°0- te60°O 240°0 OSst°O €20°0- UU 
ol 6 u ri 9 S$ 4 t é 
duLsvd Gul dV Bw. 1IV3 dO1LIV4 yOLIVG wOiadva wOldva wO1)V4 sO0i1 94 


“UOLYNALOS 4OZPDIEZ [PAL dUL4g duz AQ pauiezqg *3SaqzqGNS AuPLNgedsoA TT 1[eaAa7 
PMO] 94} 4O Si Suapuay ASUOdSaYy wal] adUul-AjuO4 vy. gO S$40ZDEY UOUMIG) AJUO4 JO KLuZeW SuLpeo] s0zde4 % 


Yeu ru 
9b °U- 
C20°U 
eeurG- 
@00°0- 
aitto- 
9ul*ou- 
190°oO 
690°O- 
BOT°E 
$90 °U- 
901° O- 
cota 
eot*u 
222° 
290°U 
t20°O 
BeO°U 
Tlotu 
6¢c0°O- 
tOl*o 
tOl°o 
f00°Q- 
HLT°O 
9tEt°o- 
e¥0°O- 
610° 0O- 
U<0*%uU 
Lt0°O 
t£0°O- 
580°U 
T0G"O 
t€0°O 
290° 0- 
2410°0O 
wloro- 
€o0*U- 
O80 oO~ 
500°0- 
960°0- 
S¢c1°o- 
b620°%uU- 
990°O 


O¢ 
oUlIv4 


220°0- 
120°O 
6$0°0 
6S50°0 
#10°O 
690°O0- 
%$0°O 
290°0- 
8e0°O 
¥21°O 
S1z°o 
$10°O 
eol°o- 
¥S0°O- 
280°O 
881°O 
060°0 
¥4%0°0- 
o£T°O- 
260° O- 
o2t°o- 
2ut°o- 
o10°O 
270° O0- 
SbIt°O- 
610°0- 
010°O 
%21°O 
€20°0- 
tta*o- 
€£0°O 
100° 0- 
4$0°0 
t20°Oo 
290°0O 
%10°O0 
6€0°O 
04%0°O- 
900°0 
9£0°O- 
Tero 
760°0- 
%10°0 


61 
wyOLIW4 


1io°o 
#00°0O 
$s0°0~ 
620°0- 
$00°0 
tzt°o 
84%0°O 
120° 0- 
¥to°o 
0s 1°O0- 
e00°0 
1€0°O- 
890°0 
9TT°o- 
192°0 
zat°o 
St0°O 
0s0°0- 
zeT°o 
090° 0- 
160°O- 
240°O 
€60°0- 
120° 0- 
o1o°o 
6F1°0 
470°O 
z€0°O0 
111° o- 
620°0 
g20°O- 
%90°0 
890°0- 
690° 0- 
€00°0- 
450° 0- 
6t2°o- 
ist°o- 
L1T0°O 
8T0°0- 
801°0 
160°0 
280°0 


BI 
yO1dV4 


020°0- 
Soto 
$80°0 
$4#0°0- 
%90°O 
420°O0- 
120°0- 
TSst°o- 
€90°0 
610 °O- 
910°0 
69T°O 
yt" O- 
Ool°o- 
o£o°o- 
ts0°o 
€80°O 
ett°o- 
491°O 
€10°o 
280°O 
€te°o 
700°O 
e20°0 
»90°0 
080 °0- 
690° 0- 
660°0- 
890°C- 
$9T°O- 
010°O0- 
220°0- 
2%1°0 
oot°’o- 
690°0- 
900°O0- 
190°0 
180°O- 
090°0- 
110°0- 
+80°0- 
o010°0 
190°O 


LI 
yOLIVGS 


%€0°O 
090°0- 
691°0 
010°0- 
€91°0- 
0z0°0 
910°0 
2€0°0 
1S0°0 
990°0- 
990°0 
910°0- 
$50°0- 
600°O 
100°0- 
Ist°o 
Lotro- 
102°0 
0£0°0- 
s€0°0 
£z21°0- 
z00°0 
9TT*O- 
890°0- 
290°0 
€10°0 
y1T°O 
SE0°0- 
140°0- 
oto°o 
290°0 
1€0°0- 
040°0 
£€0°0 
0s50°0- 
€12°0 
t20°0 
6Lt°0- 
62T°o- 
160°O 
tWw1t0- 
Z210°0 
$$0°0- 


91 
wO1dVS 


$i0°0- 
O€0°O- 
Itt*o 
#10°0- 
6€0°O0- 
TIT°O 
6%0°0 
920°0- 
0%0°0- 
890°0- 
$l0°O 
€0t°o 
doT°o 
612°0 
€£0°O- 
160° 0- 
€s0°O 
660°0- 
480°0- 
9e1°O0- 
€S0°0- 
€S0°0- 
0€0°O 
100°O- 
T#0°0- 
1eT°o 
0£0°0- 
o€0°O 
LIt°o- 
€00°0- 
$90°0- 
%£0°Oo- 
92t°0 
160°0- 
%10°O 
410°0- 
€00°o 
szo°o- 
910°O 
o2T°0- 
S60°0- 
691°0 
T21°O- 


st 
woldvd 


(panutquo9) 


e4z°0 
6S1t°O 
%90°O- 
780° 0- 
210° 0- 
Lt0°0 
€20°0- 
790 °0- 
#>I° O- 
€1l°o- 
690°0 
€»l°o 
Zzt°o 
260°0 
€*l°o- 
€st°o 
SOT°O- 
B8¥0°0O- 
6£0°0 
620°0 
911° 0- 
€00°0 
Lo9t°o- 
S10°0- 
621° 0- 
901° 0- 
090°0 
260° 0- 
860°0O 
980° O- 
*10°O 
910°0- 
600° 0- 
860°0 
€lt°o 
*00°0- 
921° O- 
660°0 
$z0°0- 
o£T°O- 
£50°0O 
1to°o- 
610°0- 


oT 
wO1dV4 


€-S J18Vl 


s00°0o 
S2t°o 
$60°O0- 
s00°0- 
€$0°O- 
Ist°o- 
L01°o 
SOE°O 
820°O 
£60°0- 
922°0- 
$s0°0 
$20°0- 
060°0 
780°O 
1st°o 
890°O 
¥80°O- 
990°0- 
€00°0 
€ot°o- 
€O0T°o- 
Zero 
0s0°0- 
920°O 
140°0- 
B10°0 
O12 °0- 
TUT°o- 
zttso- 
1Ss0°0 
d2t°o 
020°O 
¥L0°O- 
6%0°0- 
€s0°0 
080°0 
s10°0 
210°0 
700°0 
£90°0 
€20°0- 
*10°0 


el 
wOlJV4 


920 °O- 
2g0°0 
%41°O- 
09T*0- 
220°0 
2z€°O 
€00°0 
est°o 
680 °0- 
61T°O 
%40°O 
i10°O 
180 °0- 
601°O 
COT °O- 
910 °O- 
960°0O 
z€0°O 
950°0 
680 *0- 
9t0°O- 
602°O 
S10°O 
610 *0- 
8S0 *0- 
950°0- 
$S$0 *0- 
€S0°O 
€9t°O- 
%60°O 
ooT*o 
€00°O- 
190°0 
990°0O 
160°0- 
%20°0- 
120 °0- 
t10°O- 
6£0 °C- 
E%0°C 
040°0- 
L12°O- 
$90°0 


ran | 
sOLJVa 


020°0 
£10°0- 
691° 0- 
zz0°0 
S90°0- 
vorro 
z€0°0- 
410° O- 
$90°0 
8z0°C- 
090° O- 
zs0°0 
0S0°0 
6€0°0- 
020°0 
£S1°O 
0210°O 
$92°0 
€10°0- 
961°0 
$4#0°0 
990°0- 
860°0 
£00°C 
9E1° 0- 
910°C 
Ltt*o- 
€80°0- 
420°C 
910°C 
££0°C- 
861° C- 
691°0 
920°C 
951° 0- 
8st" C- 
280°0 
190° 0- 
840°C- 
110°0- 
60T°0 
09T°O 
16t°0 


VW 
wold 


tcut%u- 
rlure 
260°O 
Gs *uU 
aducu 
buutu- 
Learu- 
USi%u 
o40°9D 
Tuo*o- 
940 °U- 
svlutu 
scro 
69U °U~ 
uo0°U- 
2lu"o- 
bs070 
690° 
@tu’o 
we0°O- 
clot un 
olucu 
tle*o- 
b9u0%% 
S1U°U 
BCL°UL- 
€bO%uU- 
"OuU°d 
940% U- 
2olvu 
wu 0 
Ublvun 
eu O- 
960% O- 
ucéG*u- 
yo0"uU 
£60 °U- 
acre 
otu"a- 
260%o- 
OUTtL 
cu0ru 
leutu- 


ut 
oui dV4 


yuturd 
sd0°O- 
rbv"’d 
ecu*d 
z2i°o- 
us0°0- 
UsurO- 
av0°d 
6t0°0- 
1¢0°O- 
1Tt0°O- 
ew tb 
950°O- 
é1g°0- 
6o30°0O 
200°0- 
co00* 0- 
»l0°O 
©90°0 
scu°a- 
€50°u- 
eul*a 
6 #0°O 
oll*O- 
21 0°O- 
yur d 
sta"o- 
o2l*d 
2lu"o 
9tu°O- 
ce0°O0~ 
w0u"Q- 
Pvo°0- 
420°C 
ovord- 
tsu°O- 
0%70°0 
791°C 
4¢20°O- 
7S0°G- 
vJuG°G 
Bbiu’d 
blute- 


, 
od 


OULIVG 


294° O- 
tgvuru- 
s96"°O- 
evi o 
Scu* 
€2o°o 
910°0 
490°O0- 
“»00°O- 
vel*o 
ye 0°O- 
er o°a 
Stu*o- 
140°O 
%40°0 
éc0°O- 
vla°G- 
T00"0- 
$90°0- 
$90°O 
T21°u- 
o1T1°0 
200°O0- 
e270°0- 
4€0°O 
o10°0 
$20°O 
£60°O- 
¥t0°o 
160°0- 
190°0 
2€0°O 
$0u0°0- 
94C°O 
900°0- 
600°0- 
BbU°O- 
€10°a 
e21°o- 
990°O- 
220° 0- 
2£0°O 
9.0°0 


we 
solid 


¥¥5°O 
$00°O0- 
Tv0°o- 
Tito 
€00°0O 
100°0O 
B20°0 
280°0- 
&20°0- 
0s0°0- 
850°0- 
620°0- 
9240°0- 
#20°O 
€10°0 
€10°O 
soto 
480°0 
890°O 
020°0- 
020°0 
%S0°O- 
990 °0- 
s00°O 
620°0 
610°0- 
950°O 
680°O 
120°O 
£90°0- 
810°0- 
800°0O- 
880°O 
420°0 
etl°o- 
Tito 
4s0°O- 
%%0°O 
avt°o 
280 °O- 
$80°0- 
eL£0°O- 
£20°O- 


2 
sOlovi 


fsl*O 
asc*o 
e70*0- 
24uU°0- 
900°0 
510°0- 
9S0°0 
SOT*o- 
620°G- 
STI°O 
20T°O0- 
e20°O- 
600° O0- 
2S Eze) 
2410°0O- 
#00°O 
890°0 
Ov0"0 
260°0 
Olto- 
£20°0- 
916°0- 
810°0- 
650°0- 
450°0 
696°0O 
gco°o- 
4S0°0 
900°0- 
200°0- 
Ce£0°o- 
bou°o- 
160°O- 
$20°O 
£01°o 
910°O- 
cet’o 
2ut°o- 
200°0- 
9¢ 0° 0- 
100°0 
€sC°O 
901°O 


92 
wOsova 


eo0lT*o 
eso°a- 
70° G- 
620°0O- 
290°0- 
650° O- 
4it°o 
€10°0O 
440°O 
24€0°O 
s9T°O 
6u0°O- 
980° 0- 
€16°O 
990°0- 
2$0°0- 
9c0°0 
¥90°O 
770°O 
$¥0°0- 
610°0 
900°0 
0#0°0 
$60°0 
oTo°o- 
z10°0O 
oto°a 
050°0- 
190°0- 
¥9t°Oo- 
6¢€0°0- 
¥90°0- 
T10°G- 
T#0°O 
9£0°0- 
0%0°0- 
ool*o- 
¥c0°O 
200°0O 
8s0°O 
£e0°O 
det°o 
$20°0- 


Se 
uOidvd 


(panulzu0)) 


v10°0- 
esu°0 
990°C 
2£0°0 
eUr°o- 
¥80° O- 
180°0 
T00°O 
g10°0- 
s20°0O 
180° 0- 
1400‘06- 
$00°0- 
£¥0° 0- 
080°0- 
s0t°o- 
400°0- 
420°O 
$10°O 
€80°0- 
9co°o 
180°O 
2$0°0- 
090° 0- 
691° o- 
600°0 
z26t°o 
220° 0- 
610°0 
velco 
190°0- 
€4%0°O 
6lt?o 
$10°0- 
210° 0- 
290°0- 
1S0° O- 
020°0- 
8vto°o 
cot°o 
$50°0 
290° 0- 
c80°0 


92 
wOldV4 


€-S d1avl 


O23°O 
6S0°0O 
690 °0- 
%€0°O 
920°0- 
920°0- 
020°O 
920°0- 
s90°0 
c9Tt*o- 
$80°O 
Gtt°a- 
1£0°0O- 
200°0- 
eco°o 
110°0- 
601°O 
010°O 
QET*o- 
6%0°O 
1T00°0 
O9t°o 
210°0O 
110°0O- 
#to0°0- 
080°O0- 
020°0- 
510°O- 
426°0- 
921°0O 
080°O 
%710°O 
$70°O- 
g20°0 
901°O 
TT0°0- 
0%0°0 
$90°0- 
0s0°0O 
960 °O- 
980°0- 
290°O 
8t0°0- 


te 
wOLljvd4 


211°0 
£90°O- 
800°0O 
280°0 
Z221°O- 
210°0 
e01°O 
920 °0- 
$4#0°0- 
€10°0- 
d2t°o- 
SEt°O 
120 °O- 
%00°O 
o10°O 
100 °0- 
€S0°0- 
890 °O- 
920°0 
090°0 
906°0 
$90 °0- 
100°O 
98t°o 
620 °O 
est°o- 
220°O 
oBt°o 
280°0- 
2190°O 
760°0O 
400°0 
120°0- 
£e0°0- 
€20°O 
841 °O- 
T10°O 
930 °0- 
950 °0- 
%€G°O 
Tao °O 
£20°0 
220°O0- 


ve 
yOLIVA 


oot°a- 
tet°o 
$90° C- 
elt°o- 
0z0°0- 
st0°o 
zs0°0 
#oT°o 
sst°0o 
€40°O 
sot°o- 
020° 0- 
B10°0 
180° 0- 
120° c- 
760°0 
610°0 
0¥0°0 
#2 o- 
600°0 
€t0°o 
€80°O 
€90°0- 
600°0 
260°O 
600° 0- 
$$0°0 
BLt°o 
€21°c 
650°0- 
260° 0- 
220°0- 
290°C 
¥v0°O 
o1l’o 
#01°o- 
420°0- 
TtO*C- 
€00°0O 
6€0°O0- 
950° 0- 
010°0- 
eei°c- 


t2 
sOldvd 


tv 
eel 
Yet 
ov) 
eel 
Gel 
dtl 
Stl 
sei 
7 | 
eel 
ce) 
ved 
cts 
t2t 
B2i 
421 
921 
$2] 
621 
€2) 
221 
21 
C2t 
6ul 
oll 
iui 
oul 
sl 
oll 
€ul 
eUl 
WI 
cu 
ol 
ot 
it 
91 
St 
v1 
eI 
ral 
WM 


#T0°C- 100°0 100°0 #00°0- €20°0- 060°0- s00°0- 900°0 090°0 $s0°0 99 €¢1 
*10°0O 690°O 220°0- %£0°O ¥To°0 940°0 080° 0- 150 °O- 210° 0- 240°0- s9 243 
luG°o- oto°o 1T¥0°0- 000°0- 610°0 $4%0°0 990°O st0°O- $20°O- $90°0 9 19] 
o10°C 200°0- #t0°o Tto0°o 0s0°0- 290°0 920° 0- 620°0 0S0 °O- 600°0- €9 oof 
O’O°U- ¥10°0- #00°0O $10°O- BEOrO $s0°0- $z0°0 610°0- ovl"o $90° 0- 2 6€! 
210°u zto°o0 910°0 €00°0 9Z0°0- 120°0 820°0 €10°0- o1o°o s€0°0 19 eel 
200°O €00°0- 620°0 410°O- 250°0 2€0°0- T€0°0 ¥€0°O 9z21°O- ¥e0°0- 0 ie! 
200 °0~ 800°0- 2€0°0 200°0- €€0°0- 410°0 s10°0- ¥£0°0- €20°0 100° 6- 6S ot! 
£00°O- 120°0- 120°0- 1€0°0- z€0°0- 2z70°0- 61T°0 190°0 1€0°0O o€0°Oo 6s Sel 
110°0- 900°0 4£20°0- $20°0- 1t0°0 z10°0- %%0° 0- €40°0- 0%0°0- o10°o is set 
%00°0 $00°0- 010°0- se0°o 020°0 $€0°O L10°O0- 020°0 O£0°O- 210° 0- 9 ee! 
yO0*uU- L£10°0- €to°o z210°0 2s0°0 0$0°0- 900° 0- €10°O 6S0 °O- 6s0°0 ss 2cl 
*luco 120°0- 820°0 200°0 490°0 920°0 9€0°0- €zt°o 050 °0- T00°0 ws Te! 
80u°0- tto°o 6$0°0O- 900°0 920°0- $00°0- 6%0°O $00°0- 420°0- 0£0°0 €s ctl 
toa°u- s00°0- ¥10°O- 220 °O 100°O- 940°O- tt0°0 810°0- #00 °O- 960°C- rs) 6zt 
so0°o 110° 0- 40° O- 8€0°0- €00°0- 600°0- 610°0- 600°0 €€0 °0- €00°0 1§ ezt 
$U0°0O- 9490°0- 200°0 9€0°0 6€0°0 100°0 ¥80°O- T€0°O 200 °0 Toto 0S zt 
£00°0 200°0 900°0 z20t°o 1s0°0 sto°o0 St0°O €€0°O- 2€0°O- €20° 0- oy 921 
200°uU- 820°0- sto°o L10°0- o1o°o- 260°0 %10°O %¥0°O- 210 °0- €2t°o- oF $2} 
ZE0°O- o0°o 210°0- L10°0- 100°0- 180°O 690° 0- 9£0°0- $$0°0- veo°o- dv $21 
b20°O 100°0 Z10°0- 100°0 690°0- z0t°o- T€0°0- 280°0- STT°O- €z0°0 ov €21 
400°O- 610°0O T10°O- 920°O- T10°0- €00°0 €10°0 890°0O sto°o €10°0 SY 22) 
O10°u- 400°0 9€0°0 610°0 o10°0- 440°0 1€0°O €00°0- sto°o 6€0°0- ve t2t 
' 960°O 820°0 €20°0- 200°0- 120°0- 490°0 z7€0°0- oet°o 280°0 9€0°0- tv oz! 
sy %00°O- €00°0- 140*°0- 690°O s00°0 st0°O- 6£0°0 »T0°O 940 °0- zeorce ey 6lt 
f #00°O- 200°0 010°O- £40°O0- $s0°0- ot0°o0 £80°0- 610°0 etd °0- 4Z0°C lv eu 
010°0 gc0°0- 020°0- 6£0°0 €40°O- 200°0 s€0°o- 0%0°0- ezt°o 600°0 ov au 
t00°0- 4£20°0 %40°0 4%00°0 210°0 %20°0- ¥E0°0- OT0°O %€0°O €00°0- 6€ 9! 
1TO00°U- 400°0 190°0 €%0°0- sto°o 620°0 890°0 800°0- 820°O s80°0 Ct stl 
4t0°0- O10°0- 8S0°0- 8£o°o- €20°0 sto°o- T20°O- 910°0- 800°0- 910°C- ce $il 
&10°O o10°O0- %40°O st0°o- i10°0 T¥0°0 210°O 460°0- 610 °0- 260°0 9 eu 
¥10°O @£0°0 z00°0- 810°O *80°0 %¥0°0- 010°0 2£0°0 200°0 900° 0- Se zu 
010°0- s00°0 Ttr0°0 920°O 9T0°O- 420°0- 250°0 100°0- O£0°0 9€0°0- ve WI 
610°O TO°O- 240°O- 4€0°O- o10°o- s00°0- T10°O 6€0°0 100°0O- €00°0- ce cul 
tlu°o- 920°0- 0s0°O ss0°0 s80°0- 1z0°0 610°0 $z0°0- 990 °0- €%0°0- a3 61 
1lo°u- ¥e0°O 490°0 920°0- 190° 0- $90°0- 2S0°0- 8t0°O 910 °0- z210°0- Te eo! 
1z0°O E10°O- 400°O0- L£10°O- ¥20°0- 610°O €ro°0- %90°0 $60°0 4€0°0- Of dl 
000°0 220°0- s20°0- 210°0 610°0 $s0°0- 800°0- 64%0°O TT0°0- ¥20° 0- 62 91 
€T0°u- oto°o ¥t0°0- TEO*O- 610°O0 z21t°o o10°o €%0°O- 210 °O~ 860°0 82 $! 
910°O- g00°0- €10°0o €40°0 6%0°O 100 °C- €240°0- 820°O 610°O 2S50°0 lz “1 
10G°o 4¢0°O *t0°O- 6£0°0 6€0°0- zz0°O0- z720°0O s00°0- 200 °0 990°0 92 et 
tlo°o 610°O 600°O $20°O- os0°0o 020°0- %S0°0- TIUt*o- €80°0 tT10° 0 Se 21 
1G0*0- ¥10°0 $z0°o 4z0°0 sto°o- 920°0- 900°0 2240°O $00 °0- ssorc 92 VW 
O* 6¢ Oe ze St SE oe €€ ras T€ 
WULIVS dyULIVS yOlIV4 uOldv3 yOldV4 ¥vOl3V4 yOLJV4 vOLIv4 yOLIVS wOLIVS wr] 


(panutzuoj) €-S J1aVL 


TABLE 5-4 


~ 


: Estimated Item Discrimination Parameter @_ And Item Difficulty 
ee Parameter 5, » Proportion Correct p_ And Normal Deviate 7_, 


for Each of the Forty-Three Old Test Items of the Iowa Level 11 
Vocabulary Subtest. 


r et Discrimination Difficulty Proportion 
a Item Parameter Parameter Correct Deviate 


p 


9 ag 9 9 Yo 


0.79315 0.81740 

25 0.829 -1.000 0.73816 ~0.63768 

26 0.614 0.821 0.66624 0.42955 

27 0.594 0.340 0.56895 ~0.17370 
Rie 28 0.669 0.900 0.69162 ~0.50045 
7 29 0.867 1.077 0.75973 ~0.70543 
30 0.956 “0.557 0.64975 0.38465 

1 0.938 0.179 0.54865 0.12225 

32 0.940 -0.803 0.70897 ~0.55038 

33 0.434 22.331 0.82318 0.92755 

34 0.598 1.210 0.73266 ~0.62088 

35 0.489 0.569 0.59856 0.24962 

36 0.657 0.987 0.70601 0.54177 

37 0.351 0.577 0.42428 0.19096 

E 38 0.665 0.468 0.60237 0.25949 
39 0.333 0.676 0.58460 0.21368 

40 0.683 0.402 0.41032 0.22672 

a 0.531 0.948 0.67174 0.44472 

‘: 0.436 0.258 0.45897 0.10303 

43 0.672 0.867 0.68570 0.48370 

“ 0.143 4.175 0.27665 0.59282 

45 0.898 0.357 0.59433 0.23870 

| 46 0.612 -0.318 0.56599 -0.16617 
. a7 0.494 0.781 0.63536 0.34608 
48 0.849 0.054 0.48604 0.03500 

49 0.421 0.626 0.59602 -0.24306 

i 50 0.346 -0.250 0.53257 -0.08173 
- 51 0.664 -0.420 G.59179 0.23215 
E 52 0.640 0.217 0.45347 0.11690 
53 0.402 0.526 0.42217 0.19635 

; 54 0.573 0.126 0.47504 0.06261 
55 0.667 0.342 0.57530 -0.18988 

56 0.593 1.007 0.30372 0.51373 

57 0.370 0.398 0.44501 0.13828 

58 0.416 0.782 0.38198 0.30028 

59 0.491 0.731 0.62648 0.32254 

60 0.678 -0.170 0.53807 0.09557 

— 61 0.519 0.748 0.36506 0.34497 
62 0.938 -0.485 0.62986 0.33148 

63 0.637 0.398 0.58460 -0.21368 

64 0.818 0.042 0.51058 0.02652 

65 0.606 0.595 0.37902 0.30806 

66 0.604 0.376 0.57699 -0.19420 


SO Lilet 
wh cart (a Ae 


~24- 


item discrimination parameter, a, » and item difficulty parameter, 


b » which are given by 


- 2,71/2 
5.l bad p l= 
(5.1) a, 3! P.) 
and 
5. nan = A =~] 
(5.2) d, 1, P, e 


where Pe is the factor loading of item g on the first common 
factor. Those values thus calculated will serve as the estimated item 
parameters in the normal ogive model on the dichotomous response 
level, whose item characteristic function is given by (4.1), for each 


Old Test item. 


VI. Test Information Function of the Old Test and the Transformation 


of the Latent Trait @ to T 
The test information function, I(8) , of the Old Test is given 
by the sum total of the item information function, BC8) » which can 


be written as 


eP pos 2 -1 
(6.1) 1, (8) ey: Po (8)] [P,(€)Q,(8)] ‘ 


where PA 8) is the item characteristic function given by (4.1) and 


(6.2) Qe) zl- P(e) : 


295s 


Thus we can write 


(6.3) I(e) = t Tj(e) . 

we, g=1 7 

m It has been pointed out (Samejima, RR-80-2, Final Report) that the 
square root of the test information function, rather than the test 
information function itself, is a useful function in many ways. Among 
others, it can be used in the process of transforming the original 
latent trait to another, which has a constant amount of test 
information for the range of ability of interest, as we have seen in 
Section 3. 

Figure 6-1 presents the square root of the test information 
function of the Old Test by a solid line. We can see in this figure 
cE that our Old Test is most informative around 6 = ~O0.4 , ieee, a 
little below the average ability level of our subjects. This ts 
expected from the fact that thirty test items out of the total of 
forty-three have negative difficulty parameters, and twenty-eight test 
items have their difficulty parameters between -1.0 and 0.2 , as is 
observed in Table 5-4. In the same figure, also presented by a dotted 
line is the polynomial of degree 7 obtained by the method of moments, 
using the interval of 6 , (-5.0, 5.0) . The actual formula of this 


polynomial is given by 
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FIGURE 6-1 


Square Root of Test Information Function L1(e)j 1/2 of the 

Level 11 Vocabulary Subtest (Solid Line) And Its Approximation 

by the Polynomial of Degree 7 Obtained by the Method of Moments 
with the Interval of 6 , [-5.0, 5.0] (Dotted Line). 
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FIGURE 6-2 


Polynomial Transformation of 6 to ft. 
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ao] = 


(6.4) [1(9)}1/2 = 3.1915950 = 0.236049728 - 0.2732255002 


3 


+ 0.0262482596° + 0.0123155788° - 0001148595192 


- 0.000227876459° + 0.0000183226979/ ° 


We can see in Figure 6-1 that this polynomial and the actual square 
root of the test information function are almost identical with each 
other for the interval of 6 , (-4.0, 4.0) . This proves that the 
polynomial given by (6.4) makes a good substitute for the square root 
of the test information function, the result which is expected from 
the fact that the polynomial obtained by the method of moments is also 
the least squares solution for the specified degree of polynomial and 
interval of 9§% (Samejima and Livingston, RR-79-2). Actually, the 
method of moments was tried with four different intervals of 6 , 
dee., (4.0, 4.0) , (-4.5, 4.5) , (-5.0, 5.0) and (-5.5, 5-5) , 
and the above result showed the best fit. Those coefficients, o 
(k=0,1,2,...,7) , of the polynomials of degree 7 obtained by using the 
four different intervals of 6 are shown in Appendix as Table A-l, 
together with the coefficients of the polynomials of degrees 3, 4, 5 
and 6 obtained similarly by using those four intervals of 6, 
respectively. In this table, also presented are the original and 
revised coefficients, which mean those obtained by adjusting the area 
under the curve of [1¢9)] 1/2 for the specified interval of © to 
unity, with the midpoint of the interval and real origin of the scale 


as the origin, respectively. The revised coefficients were further 


modified to the "corrected" coefficients by readjusting the area 


-28=- 


under the curve of [1(e)} 1/2 for the specified interval of 9® to 
its real value. 
The polynomial for transforming © to Tt was obtained by ” 


(3.3) and (3.4), using the coefficients, (k=0,1,2,..-,7) , in 


me 


(6-4) and setting the two constants, Cy * 0.0 and C, = 4.0. The 


1 
resulting polynomial of degree 8 is given by 


(6.5) (8) = 0.00000000 + 0.79789874.9 - 0.029506215¢2 - 0.02276879202 
+ 0.00164051620" + 0.0006157789 16> 


= 0.0000478581276° — 0.00000813844460" + 0.000000572584 286° . 


Figure 6-2 presents this polynomial of degree 8 for transforming 4 
to T. 
We can write for the square root of the test information 


function, I*(t) , for the transformed latent trait T 
(6.6) [r#(t)]!/2 = [1¢9)]!/2 e 


Figure 6-3 presents the square root of the test information function 
of +t thus obtained by using the approximated polynomial for 

[1¢@) 1/2 given by (6.4), and the derivative of + obtainable from 
(6.5). Since the interval of 6, (-4.0, 4.0) , corresponds to the 
interval of t , (-2.44244, 2.02098), as we can see in Figure 6-2, the 
latter interval is shown by arrows in Figure 6-3. We can see that for 


this interval of t the approximated square root of the test i 
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FIGURE 6-3 


Square Root of Test Information Function L1e(2)l/4 of 

the Level 11 Vocabulary Subtest Obtained fromthe Polynomial 

Transformation of 6 to +t (Dotted Line) And Its Target 
(Solid Line). 
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information function, [1(r)] 4/2 » is almost constant, i.e., very 
close to 4.0. 

The maximum likelihood estimate of 6 was obtained for each 
individual subject from his response pattern on the Old Test items. 
Let 8, denote this maximum likelihood estimate of ability 96 for 
subject s . This maximum likelihood estimate, 6. » can directly 
be transformed to the maximum likelihood estimate of the transformed 
ability t , which is denoted by T , through (6.5). This was done 
for the 2,356 subjects, whose 6. ’s are within the interval, 

(-3.75, 3.75) . On this stage, eight subjects were excluded from our 
data, since three of them obtained positive infinity for 6. ,» three 
obtained 5.94848 , 4.92496 and 4.37878 , respectively, and two 
obtained 3.89343 . They were excluded permanently from the rest of 
the research. 

Figure 6-4 presents the frequency distribution of the 2,356 = P 
with the interval width of 0.25 . In the same figure, also 
presented by dotted and dashed lines are the polynomials of degree 3 
and 4, respectively, which were obtained by the method of moments. In 
these two cases, the interval of tT is (=1.91742, 1.95366) , taking 


the lowest and the highest values of T, among the 2,356 as its two 


endpoints. These two polynomials of degrees 3 and 4 are given by 


(6.7) ge(T) = 0.42358084 - 0.046813019T 


2 3 


- 0.13270786t~ + 0.020014202T 


and 
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and 


(6.8) e*(t) = 0.45023559 - 0.0442328531 - 0.20387563t- 


3 + 0.0221764057% , 


+ 0.018406862t 
respectively. We can see in Figure 6-4 that these two polynomials fit 
the frequency distribution very well. Hereafter, we shall call the 
situations in which (6.7) and (6.8) are adopted as the estimated density 


function, gk(T) » Degree 3 Case and Degree 4 Case, respectively. 


VII. Estimated Plausibility Functions of the Distractors 

Using the polynomials introduced in the preceeding section as 
the estimated density, g*(t) , the first through fourth conditional 
moments of +t, given c » were obtained for each individual - > 
by the formulae (3.5) through (3.8). From those results, the 
two coefficients 8, and Bo and Pearson’s criterion « were also 
computed by formulae (3.9), (3.10) and (3.11) . They are all presented 
in Appendix as Tables A-2 and A~3 for Degree 3 and 4 Cases, 
respectively. 

Table 7-1 presents the frequency distribution of the 2,356 
a “s with respect to the types of the conditional distribution of 
t , given T. » in both Degree 3 and 4 Cases. These types, 1 through 


7, are Pearson’s Types (Elderton and Johnson, 1969; Johnson and Kotz, 


1970) assigned by evaluating the values of the criterion « . We can 


see in this table that in both Degree 3 and 4 Cases more than sixty 
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TABLE 7-1 


Frequency Distribution of the 2,356 t 


with Respect to Their Pearson Types for 
the Conditional Distributions of rt 


Degree 3 | Degree 4 | 
Case 


norma] 
und. } 
und. 2 


2,356 2,356 


und. 1 : Undefined Due to Negative Even 
Conditional Moment(s). 


und. 2: Undefined Due to Negative P.N.F. 


percent of the cases belong to the normal distribution, while most of 


the others belong to the Beta distribution, i.e., either Pearson’s 
Type 1 or 2. There are some cases whose conditional distributions of 
tT are undefined, either due to negative estimated even conditional 
moments or to negative estimated conditional probability densities. 
Those subjects have to be excluded from the rest of the research. 

The above result justifies our choice of Normal Approach Method 
in both Degree 3 and 4 Cases. A close examination of Tables A~2 and 
A-3 in Appendix further discloses the fact that, in most cases where 
the conditional distributions of T belong to other types of 
Pearson’s distributions than normality the values of 8. are very 
close to 3.000 , the number which characterizes the normal 
distribution. 

Since these two sets of results are very similar to each other, 
hereafter we deal solely with Degree 4 Case. It is worth noting, 
however, that the results of Degree 3 Case would be just as 
respectable as those of Degree 4 Case, in spite of the fact that the 
degree of the polynomial approximating g*(t) is one less and as 
small as 3. A slight disadvantage of Degree 3 Case appears in the 
number of subjects to be discarded because of the result shown in 
Table 7-1, t.e., 127 compared with 53 in Degree 4 Case. 

Before proceeding further, we notice that a simple, 
straightforward estimation of the plausibility functions of the 


distractors can be made by taking the frequency ratios of the 


subgroups of examinees who selected the separate a]ternative answers, 
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which are calculated for some appropriately set subintervals of 6 . 
This was done by using thirty subintervals of 6 with unequal widths 
to make the total numbers of subjects in the separate subintervals of 
approximately equal, i.e., 78 or 79 . The results are shown in 
Appendix as Figure A-l, together with the item characteristic 
functions in the normal ogive model, whose discrimination and 
difficulty parameters are shown in Table 5-4 as ag and b, - In 
those graphs, the frequency ratios of the correct answers, A ,are 
represented by dots, and those of the distractors, B,C and D, 
are represented by dashes of three increasing lengths.* We can see 
from those graphs that the set of frequency ratios for the correct 
answer matches the corresponding normal ogive curve well in most cases. 
The operating characteristics of the four alternative answers 

were obtained for each of the forty-three vocabulary test items using 


the Simple Sum Procedure of the Conditional P.D.F. Approach combined 


with the Normal Approach Method, which were introduced in Section 3. 


eS SS 


* Because of the confidentiality of the Tests, we need to disguise the 
correct answer for each item from the reader. For this reason, the 
alternative answers, 1,2, 3 and 4 , were randomized, so that we 
shall always call the correct answer the alternative A , and the 
three incorrect answers B,C and D, throughout the rest of the 


present paper. 
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The resultant estimated operating characteristics are shown in 

Figure 7-1 by dotted and dashed lines, together with the normal ogive 
curve specified by the two estimated item parameters shown in 

Table 5-4, which is draw by a solid line. 

These results provide us with a variety of information. First 
of all, most of the nonparametrically estimated item characteristic 
functions are very close to the corresponding normal ogive curves, 
which are parametrically estimated item characteristic functions 
following the normal ogive model. Exceptions are the one for item 44 
and, to a lesser extent, those for items 39 and 49 . Secondly, most 
of the sets of estimated plausibility functions of distractors 
indicate that they belong to the Informative Distractor Model rather 
than the Equivalent Distractor Model, which means each separate 
distractor provides us with its own information. The closest 
configuration of the estimated plausibility functions to the 
Equivalent Distractor Model may be the one for item 63 . There are 
pairs of distractors which show almost identical estimated 
plausibility functions with each other in certain items, however. We 
can see them in the alternatives B and C of item 24 , B and C 
of item 29 , C and D of item 30, B and D of item 49 , B 
and C of item 50, C and D of item 52 , B and C of 
item 53, Band D of item 60, C and D of item 61 and C 
and D of item 64 . Thirdly, there are certain distractors which 


attract the examinees strongly for certain specific ranges of ability. 


Table 7-2 presents those distractors under the label, “informative” . 
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FIGURE 7-1 


Estimated Item Characteristic Function Following the Normal Ogive 

Model (Solid Line) And the Estimated Operating Characteristics of 

the Four Alternative Answers Using the Simple Sum Procedure of the 

Conditional P.D.F. Approach Combined with the Normal Approach Method, 

for Each of the Forty Three Level 11 Vocabulary Items. The Length 

of Dashes Is Shortest for Alternative A, Next Shortest for B, Longer 
for C, and Longest for D. 
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FIGURE 7-1 (Continued) 
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TABLE 7-2 
Distrectors Whose Estimated Plausibility Functions Proved 


Be 'nformative, Aid Those Whose Estimated Plausibility 
Functions Are Unusually Flat. 


 Ttem [tnformative Flat ' Item {Informative Flat 
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Note that the distractors listed in Table 7-2 are only those which 
proved to be informative from the results of the present study, in 
which the estimated operating characteristics are truncated on the 
lower levels of ability. There is no way of knowing about the 
information provided by distractors on the levels of ability lower 
than -1.6 , and it is suspected that there are many more informative 
distractors than those which the present results have disclosed. We 
shall be able to discover them by analyzing data collected upon a 
group of younger subjects in future research. Fourthly, we notice 
that there are some distractors which show unusually flat plausibility 
functions. They are listed in Table 7-2 under the label, "flat". 
Somehow those distractors attract high levels of subjects almost 
equally as they do lower levels of subjects. It will be worthwhile 

to use this information and go back to the items having such 
distractors to find out if the alternative answers selected for those 
items are suitable, or if the question itself is wel] addressed. 

In any case, the above information provides us with valuable 
resources of item analysis. Effort can be focused upon the 
improvement of each item using tuis information, until we find a more 
informative set of alternatives. 

We can see in Figure 7-1 that for some items the implicit 
orders of the distractors have been disclosed by the present research. 
For example, in item 46 , the alternative C appears to be the second 
best answer, although the order between the alternatives B and D 


is not obvious. It may be appropriate to give the graded scures, 0 , 
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1 and 2, to the combination cf 8B and D , to the alternative ¢ 
and to the alternative A , respectively, for this item. Thus the 
three fucctions, which represent the cumulative operating 
characteristics assigned to Xi, (=0,1,2) or greater, were obtained 
from the result of Figure 7-1 and those zraded scores, and are shown 
in Figure A-2 in Appendix. Similar functions were also obtained for 
sixteen other items and are shown in the same figure. In those 
seventeen graphs we can see that for items which were presented later 
in the subtest the operating characteristics fer Xe yO decrease as 

@ decreases, disclosing a strictly decreasing operating 
characteristic for the "no answer” category. 

Because of the confidentiality of the test, here we will not 
discuss the contents of the alternatives which proved to be 
informative, or which provide us with unusual configurations of the 
estimated operating characteristics, and so forth. [It might be added, 
however, that in many cases we can find possible reasons for those 
characteristics of the estimated plausibility functions, although in 


some other cases we zre rather puzzled. 


VIIL. Model Validation 

We have seen in the results of previous sections that there is 
enough evidence to support the set of assumptions we adopted in the 
present research. First of all, the normal ogive model assumed for 
the Old Test items is we!i supported by the goodness of fit of the 


item characteristic functions estimated by agsuming the normal opfve 


model to those estimated by our combination of nonparametric method 
and approach. This result supports not only the assumed normal ogive 
model but also the multivariate normality assumed for the forty-three 
response tendencies. 

It may be more appropriate to repeat the whole procedure after 
excluding such items as items 44 , 39 and 49 from the Old Test, 
which do not show very good fits of the normal ogive functions. 
Considering the fact that there are only a few items of poor fits, 
however, we cannot expect the repetition of the procedure with the 
new reduced Old Test to provide us with a substantially different set 
of results. The decision was made, therefore, against it. 

Secondly, the use of the Normal Approach Method is well 


supported by the estimated conditional moments of + , given t. 7 


which are shown in Appendix as Tables A-2 and A-3 and summarized 
in Table 7-1 in the preceding section. 

In order to proceed further in model validation, the following 
procedure was added. For each pair of items, using their estimated 


A 


normal deviates, Ye » which are presented in Table 5-4, and the 
estimated tetrachoric correlation coefficient between their response 
tendencies, the two-by-two contingency table is produced from the 
bivartate normal distritutton. The chi-square statistic of the four 
frequencies in the actual contingency table was computed by using the 
frequencies in the contingency table thus produced as the 


"theoretical" values. Since the empirically obtained normal 


deviates and tetrachoric correlation coefficient are used in obtaining 


26h 


the “theoretical” frequencies, the resultant statistic should have a 
negligibly small value, in order for the bivariate normal assumption 
to be validated. 

As it turned out, most of the chi-square values are very smail. 
Table 8-1 presents two such examples, i.e., those chi~square 
statistics for items 62 and 30 , against each of the other 
respective forty-two items. In the same table, also presented are 
two more examples whose chi-square values are largest, i.e., those for 
items 24 and 44 . In fact, item 44 has unusually large chi- 
Square statistics in its row, as is obvious in this table. It is 
interesting to note that this item is the one which provides us with 
the poorest fit of the normal ogive curve to the nonparametrically 
estimated item characteristic function, as we have seen in Figure 7-l. 
In contrast to this, item 24 provides us with a surprisingly good 
fit, as is also seen in Figure 7-1. It is worth noting that these two 
items are the items whose estimated difficulty parameters are by far 
the largest in absolute value (cf. Table 5-4). Similar results for 
the other items are given in Appendix as Table A-4. In this table, 
all the chi-square statistics greater than, or equal to, 0.! are 
circled. We can see that many items have small chi-square values 
against other items, except for those against items 44 and 24 . 

In order to investigate the relationship between the chi-square 
value and the goodness of fit of the normal ogive curve tu the 
nonparametrically estimated item characteristic function, all the 


forty-three Old Test items are categorized into four classes, i.e., 
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good, fairly good, fairly poor and poor, with respect to the goodness 


of fit inspected by the eye in Figure 7-1. Table 8-2 presents the 
item numbers thus categorized with respect to the goodness of fit and 
the frequencies of chi-square values greater than, or equal to, %.1 . 
We can see in this table that there is a negative correlativn between 
the goodness of fit and the frequency of chi-square values which are 
greater than, or equal to, 0.1 , the result which is logica!ly 
expected. On the other hand, it is worth noting that there are a 
substantial number of items which have high frequencies of large chi- 
square statistics, and yet showing good fits of the normal ogive 
functions. This appears to indicate the robustness of the whole 
procedure used in the present research. 

We notice from Table A-4 that most circles will disappear if we 
exclude those items which have thirteen or nore circles in their rows. 
In fact, there are some which will not have any circles in their rows 
at all if we solely exclude items “4 and 24 . For convenience, 
such items are categorized as Group 4. If we further exclude those 


{tems having 19 to 2! circles in their rows, f.e., items 39 , 50 


57 , 33 and 37 , then there will be more items which do not have any 


circles in their rows at all. This second group is called Group 8. 
Similarly, Group C ccensists of the items which will not have any 
circles in their rows if we further exclude the iteas having 13 to 
16 circles in their tows, i.e., items 53 , 35 , 42 , 49 and 58 . 
The rest of the ttems are categorized as Group D. Table 8-3 


oresents the item numbers of the thirty-one items classified 
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TABLE 8-2 


Item Numbers Categorized with Respect to the Frequencies of the Chi-Square Statistics 
Which Are Greater Than or Equal to 0.01, And Also to the Goodness of Fit of the 
Normal Ogive Curves to the Estimated Operating Characteristics of the 
Correct Answer Obtained By the Stmple Sum Procedure of the Conditional 

P.D.F. Approach Combined With the Normal Approach Method. 


Free 
1 


y 62 
29, 30, 31, 45, 
64 


1 25, 32, 46 

} 28, 36, 43 
55, 66 

; 26, 38 


60 

40 

51, 52, 54, 61 
41, 56, 63 

27, 65 

59 


42 


TABLE 8-3 


Item Numbers of the Thirty-One Items Categorized 
with Respect to the Groups The; Belong to, and 
Also to the Goodness of Fit of the Normal Ogive 
Curves to the Estimated Operating Characteristic 
of the Correct Answer Obtained By the Simple Sum 
Procedure of the Conditional P.D.F. Approach 
Combined with the Normal Approach Method. 


! Fairly Good 


eae enna 


29, 30, 31, 45, 48 


| 
| 
| 


26, 28, 32, 38, 40, 41, 51, 52, 
54, 59, 61, 63, 


; Total 
i Number . 
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with respect to the above categorization and the goodness of fit. 
4 There is a substantial difference between the goodness of fit of 
_ Group C and those of Groups A and B, as is expected. 
We can say that the results of this additional process further 


support our whole procedure used in the present research. 


IX. Discussion and Conclusions 

A set of data based upon the Level 11 Vocabulary Subtest of the 
Towa Tests of Basic Skills was analyzed, using the Simple Sum 
Procedure of the Conditional P.D.F. Approach combined with the 
Normal Approach Method, and the plausibility functions of the 
distractors of these items were estimated. In so doing, the same set 
of test items was used twice, i.e., once as the dichotomized Old Test 
items and then as the polychotomous test items whose characteristics 
are to be discovered. The results proved that most of the test items 
are not likely to follow the Equivalent Distractor Model, to which the 
three-parameter logistic or normal ogive model belongs. In fact, we 
have discovered many distractors which are informative, and the 
results suggest that most of these items follow the Informative 
Distractor Model. Model validation was performed from several 


different angles, and positive results were obtained. 


Methodologies involved in the present study appear to be 


promising, and they will find their usefulness in many other future 


studies. It is the author’s belief that more test data should be 


analyzed with the kind of rigorousness involved in the present study. = 


Since the results indicate that these test items belong to the 
Informative Distractor Model, the next logical step will be to 

find out how we can make the best use of the information obtainable 
from the distractors as well as from the correct answers, in order to 


increase the efficiency of ability estimation. 
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Frequency Ratios of the Maximum Likelihood Estimate 6 with Respect 

to the Choice of Each Alternative Answer, Together with the Item 
Characteristic Function (Solid Line) in the Normal Oqive Model 
ag with the Estimated Parameters. All the 2,361 Subjects Are 
: Categorized into Thirty Unequal Subintervals of 6 , Each of 
Which Contains 78 or 79 Subjects. The Mean of 6 Is Used As 
the Representative Value of Each Subinterval. The Ratios for 

the Correct Answer Are Plotted by Dots. 
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FIGURE A-2 


Cumulative Operating Characteristics Drawn for Each of the 

Seventeen Selected Items, Together with the Estimated Item 

Characteristic Function Following the Normal Qgive Model 
(Solid Line). 
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